Skip to content

Add vg lift: lift features to the graph, the gref and every path - #5110

Open
glennhickey wants to merge 1 commit into
vg-unitigsfrom
vg-lift
Open

glennhickey wants to merge 1 commit into
vg-unitigsfrom
vg-lift

Conversation

@glennhickey

Copy link
Copy Markdown
Contributor

Stacked on #5109 (vg unitigs). Review that one first; this diff is against its branch.

A tool that scans a cover (the gref paths of a GBZ, or the tigs vg unitigs writes) reports what it finds in the coordinates of the sequence it read, and those coordinates mean nothing to anything else. vg lift translates them back. It takes a BED on a cover sequence or on any path in the GBZ and writes a GAF whose records carry each feature's graph walk and four tags: sp:i: (how many non-gref paths visit the whole walk), gr:Z: (gref placement), pc:Z: (path coordinate) and an:Z: (BED name). A feature that several sequences spell at the same graph place is reported once.

The path coordinate names a place in the graph without depending on node IDs: PATH:START-END:STRAND on the first path that carries the whole feature, taking the gref path it lies along before a reference or assembly path. Paths keep their coordinates across clipped, filtered and sampled graphs, and no two places in a graph share a coordinate.

With a VCF, each ALT allele becomes a feature: the allele's walk without the first and last nodes, which sit on the reference on either side. The walk comes from INFO/AT, or from a PanGenie variant ID for a single-ALT record without one. The output VCF carries each allele's gref placement in INFO/GR and its path coordinate in INFO/PC.

Where an allele runs along a gref alt contig and differs from it, an insertion carrying SNPs for instance, each difference becomes a biallelic record on that contig, written after the calls the way vg deconstruct writes nested records. A haplotype gets 1 where its called allele has the variant, 0 where the allele reads that stretch of the contig without it, and . otherwise, so frequencies are among the carriers of the contig. GQ is the lowest GQ of the calls behind the genotype. This is what lets a caller's output be read in gref coordinates, where the alt contigs hold the sequence the reference lacks.

With --depth it writes how many distinct haplotypes read each base of the gref paths as bedGraph, including runs of depth 0, optionally within regions.

Two limits worth knowing before running this on whole genomes:

  • VCF mode holds every ALT allele's feature in memory for the one pass over paths, so a genome-wide deconstruct VCF wants running per chromosome.
  • BED-on-path mode keeps the steps of every path named in the BED, which is large for a BED spanning many assemblies' contigs.

Tested on macOS: make -j8 clean, scripts/lint.py clean, and prove t/59_vg_lift.t t/58_vg_unitigs.t t/11_vg_paths.t green (228 assertions). 59_vg_lift.t pins exact records for nested SNPs in a called insertion, allele counts among contig carriers, a walk read backwards giving the same records, GQ taken from the calls, phased and unphased genotypes, a sample without GQ, contig-end cases, and --depth rows including off-gref bases.

🤖 Generated with Claude Code

A tool that scans a cover (the gref paths of a GBZ, or the tigs vg unitigs
writes) reports what it finds in the coordinates of the sequence it read, and
those coordinates mean nothing to anything else. vg lift translates them back:
it takes a BED on a cover sequence or on any path in the GBZ and writes a GAF
whose records carry each feature's graph walk, how many non-gref paths visit
the whole walk (sp:i:), its gref placement (gr:Z:), its path coordinate
(pc:Z:) and its BED name (an:Z:). A feature that several sequences spell at
the same graph place is reported once.

The path coordinate names a place in the graph without depending on node IDs:
PATH:START-END:STRAND on the first path that carries the whole feature, taking
the gref path it lies along before a reference or assembly path. Paths keep
their coordinates across clipped, filtered and sampled graphs, and no two
places in a graph share a coordinate.

With a VCF, each ALT allele becomes a feature: the allele's walk without the
first and last nodes, which sit on the reference on either side. The walk
comes from INFO/AT, or from a PanGenie variant ID for a single-ALT record
without one. The output VCF carries each allele's gref placement in INFO/GR
and its path coordinate in INFO/PC.

Where an allele runs along a gref alt contig and differs from it -- an
insertion carrying SNPs, say -- each difference becomes a biallelic record on
that contig, written after the calls the way vg deconstruct writes nested
records. A haplotype gets 1 where its called allele has the variant, 0 where
the allele reads that stretch of the contig without it, and . otherwise, so
frequencies are among the carriers of the contig; GQ is the lowest GQ of the
calls behind the genotype. This is what lets a caller's output be read in gref
coordinates, where the alt contigs hold the sequence the reference lacks.

With --depth it writes how many distinct haplotypes read each base of the gref
paths as bedGraph, including runs of depth 0, optionally within regions.

Whether a VCF belongs to this graph is checked by looking for each ALT allele
in the sequence of its walk, allowing for the padding base at either end. A
one-base ALT against a longer REF is only that padding base, since the
trimming some panels do takes the allele's own sequence out of the columns,
and nothing is then left to test. A GT that names an ALT allele its record
does not have is read as the reference and counted, since reading it that way
is a guess.

Features are read as a VCF only when the path names a regular file that
htslib reads as one. Probing a FIFO would read its first block into htslib's
detection buffer and throw it away on close, leaving whoever opens the path
next in the middle of the file, and a VCF has to be a regular file here in any
case, since the lift reads it twice.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants